Different values of hyper-parameters often prove to be significant drivers of model performance and are expensive to tune and mostly task specific. Hyper-parameters play such a crucial role in modeling architectures that entire research efforts are devoted to developing efficient hyper-parameter search strategies (Bergstra et al., 2013; Nguyen et al., 2019; Henderson et al., 2018; Van Rijn and Hutter, 2018; Probst et al., 2019). The set of hyper-parameters differs for different learning algorithms. For instance, even a simple classification model like the decision tree classifier, has hyper-parameters like the maximum depth of the tree, the minimum number of samples to split an internal node and the criterion to use for estimating either the impurity at a node (gini) or the information gain (entropy) at each node. Ensemble models like random forest classifiers and gradient boosting machines also have additional parameters governing the number of estimators (trees) to include in the model.